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This paper discusses the results of research on 
learning LOGO programming language by two groups of fifth grade 
students who were taught using either a structured tutorial mode or 
an unstructured discovery method. Previous results showed mastery 
differences between the structured and unstructured teaching methods 
with the results favoring the structured approach. Using classroom 
observational techniques, the present study examines how students in 
the LOGO classes compare with students in traditional situations for 
such factors as time on task, discipline, and organization. 
Additionally, this research asks whether techniques other than 
mastery measures can be used to differentiate LOGO classes taught by 
structured and unstructured techniques, and whether there are 
observable classroom behaviors that differ in classes taught using 
the two different teaching methods. Findings indicate that while 
traditional methods of classroom observations may be appropriate to 
differentiate traditional classrooms from LOGO classrooms, they may 
be inappropriate to differentiate various LOGO learning environments; 
and that, while it may be possible for LOGO classrooms to look as if 
adequate learning is taking place, mastery may not happen. It is 
suggested that research in LOGO to specifically measure mastery is 
needed before claims of transfer can be taken seriously. (JB) 
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Abstract 

This study examined whether a widely used classroom observation 
instrument would differentiate fifth grade classrooms in which 
Logo was taught using a structured or an unstructured teaching 
method. The Logo classrooms were also compared to traditional 
classrooms. It was previously demonstrated that the structured 
group demonstrated significantly better mastery of Logo. Although 
there were differences between the Logo classrooms and 
traditional situations, no differences were found between the two 
different Logo environments even though the students in the two 
Logo classes showed different degrees of Logo mastery. The 
findings have specific implications for studies of transfer. 
Since observation of classroom behavior does not differentiate 
between efficacy of learning, specific measures of Logo mastery 
must be obtained before transfer claims related to Logo are 
made. Additionally, there is a need for development of 
monitoring systems to measure learning in Logo classrooms, as 
well as for the modification of traditional observational systems 
if we desire to make observationally-based statements about 
different Logo learning environments. 
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Different Logo Learning Environments and Mastery: 
Relationships Between Engagement and Learning 

Logo is receiving an increasing amount of attention from 
both educators and parents. One reason is that it is inexpensive 
and has become available for a wide variety of microcomputers. 
Another is that Logo is simple enough for relatively young 
children to learn, yet can be challenging even for adults. In 
addition, educators such as Papert (1980) have argued that in the 
course of learning Logo, students develop ways of thinking and 
solving problems that will help them in other areas. This claim 
about potential benefits of Logo has generated conniderable 
enthusiasm, in part because of a growing concern with the need 
for schools to develop effective thinking and problem solving 
skills (e.g., see Bransford & Stein, 1984; Shipman, Segal & 
Glaser, in press; Segal, Shipman & Glaser, in press). However, 
wholesale, uncritical adoption of Logo is potentially dangerous. 
Little is known about many aspects of Logo, including such 
important issues as the effect of different methods of teaching 
Logo, mastery of the language across different instructional 
situations, and possible differences between what takes place in 
Logo classrooms compared to traditional classrooms. 



Several authors have cautioned against unbridled enthusi 
about the effects of computers on learning. ^As Maddux (1984) 
notes, "... uncritical and unrealistic optimism [for 
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microcomputers in general] is doomed from the start and could 
result in a backlash reaction against educational computing" 
(p. 36). Similarly, David Morsund's (1983-84) editorial in The 
Computing Teacher emphasizes the dangers of unrealistic 
expectations about Logo. He states: 

It feels to me like Logo has been oversold. 
Marketing experts have done their Job, but that isn't 
what has oversold Logo. Educators have done it to 
themselves. In looking for "the answer" in computing, 
these teachers have latched onto Logo. It obviously is 
part of the answer, but transforming a partial solution 
into a panacea is damaging, both to education and to 
the potential of Logo. (p. 3) 

One of the consequences of generating unrealistic expectations 
about educational innovations is that, ultimately, the good may 
be discarded along with the bad. People may be diverted from 
looking into potential benefits and, instead, focus only on the 
fact that Logo may have fallen far short of its initially-stated 
(but possibly unrealistic) goals (e.g., see Euchner, 1983). 

In this paper we focus on the level of mastery of the Logo 
language across two methods of instruction, and on the need for 
clear description of what takes place in Logo classrooms compared 
to traditional academic classes. Documenting mastery and 
observing general classroom behaviors in Logo instruction 
warrants special study for at least two reasons. First, 
different instructional techniques in Logo may result in 
different amounts of learning. A popular method of instruction 
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in Logo involves the open, discovery method of teaching. The 
reason for teaching Logo in this way is based on observations of 
children who appear to be successful at learning generalizable 
thinking skills when taught Logo in this type of environment 
(Papert, 1980). These observations have led to claims that the 
open method of teaching Logo is an effective one. While this may 
be true, it is nevertheless possible that the uses of other 
methods to teach Logo may result in more effective learning 
outcomes. Research in other areas indicates that instructional 
methods can be a major influence on what is learned 
(Arbitman-Smith, Haywood & Bransford, 1984; Bransford, Stein, 
Arbitman-Smith <Sb Vye, in press; Brown, Bransford, Perrara & 
Campione, 1984; Pelclos, Bransford & Haywood, 1984). 

A concern with mastery and with observation of classroom 
learning is also important for the issue of whether Logo 
facilitates the development and transfer of problem solving 
skills. Studies of transfer must specify the method of teaching 
Logo, plus provide some measure of whether learning of Logo has 
actually taken place. In recent research studies using the open, 
discovery teaching method, students who received training in Logo 
failed to show any transfer to tasks involving planning and other 
problem solving (Pea, 1983; Pea & Kurland, 1983). However, most 
studies have either failed to assess the degree to which students 
learned Logo programming or have found that most students' 
abilities to understand and generate Logo programs was 
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considerably lower than anticipated (Pea & Kurland, 1983). It 
seems clear that if students fail to learn Logo in the first 
place, one should not expect transfer to occur. 

Studies conducted at Vanderbilt University with fifth grade 
students of varying academic ability show that different methods 
of teaching Logo have important effects on mastery (Delclos, 
Littlefield & Bransford, in press; Delclos & Littlefield, 1984). 
All students received one hour of Logo instruction per day for a 
total of 25 school days. None of the children had received any 
previous instruction in Logo. Two different methods of teaching 
Logo were used. The unstructured method was based on a discovery 
learning model. In this method, the instructor introduced new 
Logo commands or concepts, completed one or two concrete, 
illustrative examples, and related new material to the previous 
day's work. Students then completed an example which made use of 
the new material. The remaining class time (approximately 40 of 
the 60 minutes) was unstructured, allowing students freedom as 
long as they were using Logo. The teacher's role during the 
unstructured, exploration time was to observe and answer 
questions. 

The structured method was based on a structured tutorial 
model. Each day the students used an interactive tutorial 
computer program which presented the same material and used the 
same examples as those used in the unstructured method. However, 
when the lesson was completed students were given a sheet 
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containing sample figures that made use of the presented 
material. Students were instructed to select one of the sample 
figures and to write a program that would draw it. The teacher's 
role was to ensure that the students worked on the assigned task, 
and also answered questions as they arose. 

Three kinds of programming mastery were assessed across both 
instructional methods: mastery of Logo commands, comprehension 
mastery, and the ability to produce Logo programs. Command 
mastery refers to knowing the meaning of specific commands and 
the requirements for their use (e.g., knowing that PD means 
FORWARD and that a number must be used with this command). 
Comprehension mastery refers to an understanding of what given 
commands will produce, i.e., the ability to predict the figure 
that would result from a given set of commands. The production 
of programs aspect of mastery refers to the ability to write Logo 
programs which produce specified figures. 

The results of the mastery tests indicated that the method 
used to teach Logo had an effect on what was learned. 
Specifically, there were no differences between the two 
instructional methods in command mastery, but there were 
significant differences in comprehension mastery and in the 
ability to produce Logo programs. Both groups knew the commands, 
but students in the unstructured group were less able to predict 
the effects of commands and use then in writing programs. Given 
this evidence that the method of instruction affects the degree 
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to which Logo programming skills are acquired, it becomes 
important to ask how obvious it is to outside observers that 
students are or are not learning Logo. It may be the case that 
students who are taught Logo under certain instructional 
conditions (e.g., open instruction) may appear to learn Logo 
without really acquiring the level of understanding needed to 
predict and produce Logo programs. Further, our traditional 
methods of observing traditional classrooms may not be 
appropriate if the goal is to describe what is taking place in 
classrooms where students spend a great deal of time interacting 
with computers. If these two possibilities are true, then 
teachers, administrators and parents will need to be especially 
careful in monitoring the progress that students seem to make. 

This paper presents and discusses the results of further 
research on learning Logo by the two previously-mentioned groups 
of students taught using either the structured or unstructured 
methods. As noted earlier, mastery differences were shown 
between the structured and unstructured teaching methods with the 
result favoring the structured approach. The present study 
examines how students in the Logo classes compare to students in 
traditional situations in factors such as time on task, 
discipline and organization. Additionally, this research asks 
whether techniques other than mastery measures can be used to 
differentiate Logo classes taught by structured and unstructured 
techniques; are there observable classroom behaviors that differ 
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in classes taught using the two different teaching methods? Such 
results are important for separating engagement time from 
learning time. 

Method 

Subjects 

Subjects for this study were two classes of fifth grade 
students. Subjects were varied in their level of academic 
achievement and included students who were academically less 
successful as well as academically successful as defined by 
achievement tests and teacher ratings. Both groups were equal in 
their initial knowledge of Logo. None of the students had prior 
experience with Logo. The groups were also equal in their general 
knowledge of microcomputers and their use. 

Instructional Techniques 

All subjects received the same number of hours of 
instruction: one hour per school day for 25 days. The two 
different instructional procedures discussed earlier were used. 
Twenty subjects were taught Logo using the structured approach 
and 18 were taught using the unstructured method. 

Instrument and Data Collection 

Both fifth-grade classes were observed using the Stallings 
Observation System (SOS) (Stallings, 1983). This instrument has 
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been widely used during the past 10 years: 

The SOS has been used in all grades from 
kindergarten through twelfth grade, in nearly all 
subject areas, in all types of school districts 
(including urban, suburban and rural districts) and in 
all parts of the country. It has been used as an 
evaluation tool and as a guide to improve teachers • 
in-clas8 instruction. Essentially, it is diagnostic, 
prescriptive, and evaluative. (Stall ings , 1 983, p. 
8). 

In short, the instrument used in this study is a widely used 
and respected observational measure that allows detailed 
description of both teachers' and students' classroom behavior. 
Further, criterion norms have been established that indicate 
optimal levels of behavior to which the class being observed may 
be compared. For example, the criterion for behavior statements, 
that is, the statements that a teacher makes to control student 
behavior in order to keep them on task, may be 15^ (15^ of the 
teacher's statements are used to control student behavior). If a 
teacher's behavior statements rise to 30^^ then more time is 
being used for controlling students than is optimal for learning 
to occur. The criterion levels have been carefully developed and 
extensively tested. They are one reason for the popularity of 
the Stallings' instrument; most classroom observational systems 
do not provide a "yardstick" to which to compare observed 
behaviors. 

In the SOS, criteria have been developed for several 
different types of classes, from academic areas through shops 
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courses. The criteria used in this study represent classroom 
interactions and time allotments appropriate for academic, 
intermediate-grade (grade 4-6) classrooms containing students of 
average ability. 

The SOS is also stringent in its observational technique. 
Observers must be trained and undergo checks to ensure at ler.at 
85^ interrater reliability. Three independent observers observe 
a class three times during data collection. Data is gathered 
using both a "snapshot" and a "five-minute interaction" 
observation. The snapshot records everyone in the classroom at a 
given moment and tabulates each person's activity and with whom 
they are interacting. This is done five times during a one-hour 
class period. Observation takes place over three different days, 
resulting in 15 snapshots, and provides valid and stable data 
(Stallings & Kaskowitz, 1974). The five minute interaction 
describes and quantifies the interactions among students and 
teacher. These data are also gathered across five 
regularly-spaced time periods in each class session. 

In this study, both the structured and the unstructured 
classes were observed by three different observers on three 
successive days. The observers did not know which method of 
instruction they were watching. 

Results 

The Stallings Observational System provides a method for 
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coding a large number of possible classroom behaviors. In 
analyzing the data for this study, several different categories 
of behaviors were targeted for comparison with the norm, or 
•'criterion". These were 



Teacher's Academic Statements. 

- Organizational Statements. 

- Behavior Statements. 
Students' Academic Response. 

- Students' Academic Comments. 

As required by the SOS, each interaction was coded, and a 
percentage of the total number of interactions was determined for 
each of the above categories. 

Further, the following areas were targeted for examination 
of teachers' and students' time allocations: 

- Teacher involvement with one student. 
Teacher involvement with whole class. 
Students involved in practice/drill. 
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Students receiving instructions. 
Students involved in social interaction. 

- Students involved in discussion/review. 

- Students being disciplined. 

- Students uninvolved. 

The SOS indicates the percentage of instructional time spent in 
each area. 

The instrument provides a method of coding interactions 
across four general areas and a wide variety of subcategories. 
In order to classify classroom behaviors for the purposes of this 
study, the four general classifications and two of the 
subcategories (students' academic responses and comments) of 
potential interactions were determined a priori to be the 
categories of interest. The eight time on task categories were 
similarly chosen. 



INSERT TABLE 1 ABOUT HERE 



Table 1 presents percentage scores for the interactions and 
time on task observations. Table 2 presents the results of 
chi-square analyses performed on the percentage figures across 
the structured, unstructured, and criterion groups. Normally, as 
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in most traditional measures of classroom behavior, the 
percentage figures are simply compared to the criterion in a 
descriptive manner. The chi-square analysis was performed in 
order to make statements about differences between the two 
methods of teaching Logo, as well as to compare the Logo 
classroom to a traditional academic classroom. 



INSERT TABLE 2 ABOUT HERE 



In interpreting Tables 1 and 2, decisions concerning whether 
it is better for the observed data to be at, above, or below the 
criterion levels must be based on a consideration of the nature 
of each specific category and our current knowledge of good 
educational practice. For example, it is appropriate to have 
academic statements in a classroom where learning is the goal. 
In general, therefore, it is good if the criterion level for 
academic statements is exceeded. Conversely, in the behavior 
statement category where the teacher attempts to establish 
discipline or modify behavior, a smaller number of statements 
relative to the criterion may be viewed as appropriate. This is 
also true of the other categories (organizing/management 
statements and social statements) which take time away from 
direct, academically-oriented activities. 

The numbers in Table 1 quantify the type of interactions 
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observed across the four general categories. Criterion levels 
are also presented. Regardless of the method of teachin^^ Logo, 
academic statements were over 17 percentage points higher than 
the criterion. In all of the other general categories, the 
number of interactions was significantly lower than the 
criterion. These data indicate that the Logo group in toto spent 
more effort on academic or instructional behaviors than on 
non-instructional behaviors such as social interactions, behavior 
modification and organizational/management behaviors. In effect, 
these results suggest that Logo classrooms exhibit more 
learning-oriented interactions than do normal classrooms as 
represented by the criterion. 

In the two categories involving student responses, "student 
responds academically" and "student academic comments", the 
observed data are within 1.6 percentage points (averaged across 
the two Logo groups) of the criterion. The fact that no 
statistically significant differences were found when the two 
instructional groups were compared to the criteria indicates that 
the number of students' academic statements and comments appear 
to be optimal for learning (see Table 2). The number of students' 
statements and comments are neither disproportionately large, 
indicating that they are taking place at the expense of other, 
potentially desirable behaviors, nor small, indicating that 
other, potentially undesirable interactions are not domina,ting 
the Logo classrooms. 

ERIC ^ 
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Averaging across the two classes, teachers were 25 
percentage points above the criterion in the amount of time spent 
working with one student, and were over 22 percentage points 
below the criterion in amount of time they spent with the whole 
class. The chi-oquare analyses indicate that these are 
statistically significant differences. According to the 
criteria, slightly more time should be spent in whole group 
situations than in individual situations. Nevertheless, the 
observational data indicate that significantly more time is spent 
in individual interactions in the Logo classrooms. When the 
nature of the material in the Logo classes is considered, it 
becomes reasonable to expect teachers to spend more time with 
individuals than with the whole group. Furthermore, the large 
amount of hands-on practice required to learn Logo necessitates a 
smaller amount of time spent on formal, whole group instruction, 
and a greater amount of time allowed for practice than is 
normally found in traditional classrooms. This is reflected in 
the observational data. The remainder of teacher-time was spent 
in non-student related activities, such as trouble shooting on 
equipment , organizing software , etc. 

The remaining section of Tables 1 and 2, Time Allotments, 
presents time-on-task measures in the two Logo classrooms. In 
each category, the data indicate that there are clear differences 
between the Logo classes and the traditional classroom 
represented by the criterion. For example, the students spent 
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more time on practice, as well as on discussion/review of the 
concepts. They spent significantly less time receiving 
instructions. Their on-task behavior, therefore, was 
significantly better than the criterion, while their time spent 
on off-task behavior, such as being disciplined, being uninvolved 
or in social interactions, was essentially at criterion, 
optimally placed at 0^ time. 

Thus far the discussion has centered around the differences 
between the Logo classes and the criterion. In all cases, the 
Logo classes either match the criterion or move in the "best" 
direction, i.e., are significantly higher or lower, as 
pedagogically appropriate. The remaining discussion centers 
around the second question raised in the introduction: Are there 
observable differences between Logo classrooms which use 
different instructional methods? 

As noted in the introduction, previous research reported on 
this group of students has clearly indicated significant, 
measurable differences in learning outcomes across structured and 
unstructured Logo environments. The question, therefore, is 
whether there are observable differences between such 
environments that can be detected by an observer while 
instruction is taking place. In this study, both methods of 
instruction in Logo resulted in classroom environments that 
looked desirable in relation to traditional classrooms(i.e. , had 
fewer behavior statements, more time on task, etc.). Therefore, 
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a lack of observable differences between instructional methods 
could result in the illusion of learning when in fact the best 
method of instruction was not being used. The chi-square results 
in Table 2 indicate that no statistically significant differences 
were found between the two Logo instructional situations. 

Discussion 

This study has several implications. First, although the 
Stallings Observational Instrument differentiated between Logo 
and traditional classrooms as represented by the criterion, there 
were no observable differences found between the two Logo 
classes. Traditional methods of classroom observation may 
therefore be appropriate to differentiate traditional classrooms 
from Logo classrooms, but may not be appropriate to differentiate 
various Logo learning environments. 

One reason for this possibility is that observations across 
Logo environments may not be appropriately coded on traditional 
measures. Although students in the unstructured group were 
almost always on task, they frequently engaged in trial and error 
behavior (e.g., "I'll see what this does") rather than first 
planning a design and then attempting to program. In contrast, 
students in the structured group were frequently prompted to 
plan, program and debug. In short, the quality of specific 
activities involving the microcomputer was not assessed with the 
traditional observational measure. The degree to which students 
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use microcomputers in ways that optimize learning thus needs to 
be incorporated in traditional classroom observational 
instruments* 

A second important implication of the present results 
involves studies dealing with the issue of transfer. As noted 
earlier, most such studies have used the open instructional 
method. Their general finding of no significant transfer may be 
due to inadequate mastery of Logo rather than to a lack of 
transfer effect. This possibility would not be very probable if 
observers in Logo classrooms could easily see that inadequate 
learning was taking place under conditions of open instruction. 
Based on the present data, however, it seems clear that Logo 
classes can look excellent despite the fact that mastery may not 
be taking place. An important implication is that reatearch in 
Logo must specifically measure mastery before claims of transfer 
(or a lack thereof) can be taken seriously. Research on 
different instructional procedures for teaching Logo can have 
important implications for the teaching of other subjects (e.g., 
science, mathematics) as well. 
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Sttttititi 0.30 

4. Soclil 

Sttttitnti 0.30 

6. Stadmt 

Acidiilcillj 17.80 

6. StQdtiit 
Aetdiilo 

Cottmti 4.10 



07.00 08.30 



1.60 



0.10 



0.20 



1.30 



0.20 



0.10 



14.40 16.10 



6.20 



4.60 



80.00 



16.00 



1.00 



2.00 



16.00 



3.00 



Time 

Allotment! 



1. Titcbir 
With om 

Stadtiit 66.70 

2. Ttftcbtr 

Wltb All 23.30 

3. Stadiat 
frtctlci/ 

Drill 30.30 

4. StudiBt 

RlCllTlof 

loitrtctloiii 3.60 

6. Stadmt 
Social 

Ittirtctloi i.eo 
6* Stodtiit 

OitCQflflloi/ 

R*Tlif 24.80 

7. StQdifit 

UnliiTolTid 0.00 

8. StQdmi 

BtlDf 

DiielpUitd 0.00 



63.30 60.00 



23.30 23.30 



42.80 36.60 



7.10 



0.00 



13.20 



0.70 



0.00 



8.40 



0.00 



18.80 



0.40 



0.00 



36.00 



46.00 



4.00 



26.00 



0.00 



10.00 



0.00 



0.00 



ERIC 



|ot£. !!• tlrtt fotr litirietloi eitfforlii •leoipiti ill o^iirrid 
litiraetloic iid ioiil lOOl. Ut rfiililif ito litirietloi titfforlfi 
■ri ••^citiforlii 9ho99% I yrlorl Iroi ■ ^ool of ^offUlf 
iittriotUii. Uiekir dot. 1 i 2) iid •iidiit dot. 8 ft 4) ilit 
■Uotaiiti io lot ioiil tool ^•eitfi tlii tartlitH ti iMtrtotloa 
<t.f., ttaekir ■■•tfrlif door) lol toitidfrtd. 
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Tmbit 1 

Chi Squart Uctutii! InierAcilont and Tltnt Alloimenlt 



Oomprrlton 



Interaction 
OatcKory 



Unntructcrcd 
vt. Structured 



Unttructurtd 
vt. Orlterlon 



Structured 
VI, Orlterlon 



1. Aetdtalc 

Stftttatatt 0.00 

2. OrfioiflBf/ 

MiDiflfif 

Stitiaiati 

8. BihtTlor 

5ttt»B»0tl 

4. Social 
StBt»a»Btt 

5. 5tad»Bt 
ActdiaictUy 0.02 

6. StedtBt 
Acftdtalo 

COBB»Dt» 



1.70 



0.00 



1.60 



0.00 



Time 

Aliotmenti 



id 

ERIC 



1. Tttchtr 
With oat 

StQdtat 0.80 

2. t»tch»r 

ffitk All 0.00 

8. StOdiDt 
Prtctict/ 

Orill i.eo 
4. StadiDt 

R»C»iTiB| 

Ia»traetioB» o.60 

6. StQdtat 
Sociil 
lattrtctioB 

6. StadtBt 
Oi»ca»»io&/ 

ll»Ti»e 2.80 

7. StadtBt 

UBiBT0lTt4 

t. Stidtat 

BtiBf 

Di»ci^liB»4 



7.60 



6.80 



18.70 



14.60 



8.20 



t.eo 



•.80 



80.60 



• .•0 



0.20 



• f i 0.06 : 

r < 0.01 

Jott. CU-t(|«iri TBliti 9%t% ealenlatid aeiDC Yatte* eorreetioa. 
Cki-e(|Oire aiilytii tin aot ptrforaid if aort tiaa oat eill*i 
oWarfad pire»ati|i talai tat liit tkaa 6. 
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